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The gender effect on the evaluations of multimedia textbooks 



Introduction 

Both the boys and girls use educational software in schools, but they have 
different preferences about the characteristics and the content of educational software 
(Caftori, 1994; Joiner, 1998). 

Previous studies (Kliman, 1999; Passig & Levin, 2000) have revealed that 
educational software is often designed for boys and not for girls. This kind of 
software does not motivate girls and it may explain why girls perform as well on 
educational software as boys. Therefore, a question arises which programs are 
preferred by boys and which ones by girls and which programs are easier to navigate 
by boys and which ones by girls. 

This research focuses on the students’ evaluations of manipulation and 
computerised assessment. These two fields were chosen because manipulation instead 
of turning the pages and computerised assessment are two of the most important 
differences between the traditional and multimedia textbooks. 

Literature review 

Different researches have studied the attitudes of boys and girls towards 
computers (McGrath & Thurston, 1992; Rattanapian & Gibbs, 1995; Teh & Fraser, 
1995; Young, 2000). There are some studies are about preferences about the 
educational software (Caftori, 1994; Kliman, 1999) and preferences about design of 
educational software (Joiner, 1998; Passig & Levin, 2000). 

Some research found that boys have greater interest in computers than girls both 
at home and at school (Rattanapian & Gibbs, 1995). The other researchers have 
declared, that there were no significant gender differences in boys’ and girls’ attitudes 
towards computers (Teh & Fraser, 1995). Kay (1992) has claimed that there are fewer 
differences in computer attitudes and use among preschoolers and primary school 
students than older students. One piece of research (McGrath & Thurston, 1992) 
found out that girls like computers more than boys. Maybe one reason is educational 
software, which is not developed according to girls’ needs and preferences. 




Caftori (1994) investigated which educational software is preferred by boys and 
which by girls. She found out that boys choose more difficult programs than girls. 
Author considered the programs difficult, when a great deal of trivia had to be 
memorized. Also boys liked more aggressive programs. Kliman (1999) analysed the 
educational software. She asserted that many computer games are stereotypically for 
male audience. These programs are violent, aggressive, with primarily male 
characters, and focused on competition. 

Joiner (1998) compared preferences of boys and girls in four educational 
programs: Pirates (all characters are men), Princess (all characters are women), 
Honeybeards (neutral by gender) and Blocksworld (abstract program). All these four 
programs were different versions about one of the educational program. It became 
evident that boys preferred the program named Pirates and girls preferred the program 
named Princess. 

Passig and Levin (2000) have found that compared to girls, boys gave attention 
to navigational support. They wanted to know how to continue, how to go backwards, 
they preferred variety of choices. Girls paid attention to learning interface and dealing 
with colour and appearance. 

Method 

Fifty-four students (21 boys and 33 girls) from four schools in Estonia 
participated in the experiment. Their age was between 15 and 16. All the groups were 
of mixed ability. 

Six multimedia textbooks were chosen for the study: mathematics, chemistry, 
geography, Estonian language and 2 textbooks of history. Six units of each textbook 
(except mathematics, in which 5 units were taken) were used in the experiment. These 
units of the multimedia textbooks were quite different in their structure and features. 

The study took place during a period of over 8 months. All these 35 units of the 
multimedia textbooks were presented to the students. Students worked with 
computers independently with every unit. After learning each unit, we asked the 
students to evaluate the ease of the manipulation and fitness of assessment in a 10- 
point scale. 

At the same time, we analysed the units of multimedia textbooks. Forty-six 
characteristics were about the multimedia textbook manipulation, such as number of 
menus, percentage of terms in menus and submenus, search capability, navigation 




possibilities, number of commands, buttons and icons etc. Twenty-five characteristics 
specified questions and responses in the unit such as the modes of questions, replying, 
feedback and hints. 

The values of the characteristics of each unit were found by using strictly fixed 
rules. Some of the characteristics were on alternative scales and expert opinions in a 
5-point scale (-2 to +2) were used for evaluating the three characteristics of the 
manipulating and one characteristic of the assessment. 

Results 

The statistical package SPSS 11.5 for windows was used for data analysis. We 
calculated coefficients of Speraman rank correlation between the evaluations of boys 
and girls and the units' characteristics to find out the characteristics of multimedia 
textbooks, which make the manipulation simpler as for boys so for girls and 
assessment also more suitable for both sexes. The most important Spearman 
correlation coefficients are given in Table 1 and in Table 2. We used also ANOVA for 
comparing different programs and Mann- Whitney U-test for comparing boys and 
girls. 



Table 1 Correlation coefficients between the evaluations of plainness of manipulating 
and the characteristics of program units. 



Characteristic 


Mean 

value 


Standard 

deviation 


Correlation 
with boys' 
evaluation 


Correlation 
with girls' 
evaluation 


No 


Name 


109 


Guidelines on the title page 


.38 


.49 


.03 


.52** 


131 


Number of the levels in the menus 


.51 


.51 


-.22 


-.41* 


138 


Number of key-combinations 


6.17 


3.19 


-.05 


- .42* 


141 


Percentage of terms in the words 
of sub-menus 


44% 


26% 


- .30 


- .42 * 


143 


Search capabilities 


.34 


.48 


- .29 


- .57** 


148 


Number of commands (icons, 
buttons, key-combinations etc) 


22.29 


12.82 


-.08 


-.36* 


149 


Percentage of familiar commands 


56% 


38% 


.27 


.42* 


151 


Percentage of familiar icons 


64% 


39% 


.32 


.40* 


158 


Percentage of hyperlinks with 
marking 


59% 


40% 


.12 


.53** 


162 


Number of possibilities for 
navigation 


1.8 


.68 


- .29 


_ 49 ** 






167 


Attractiveness of the realization 


.20 


.60 


-.32 


-.37* 


202 


Percentage of screen area for text 


58% 


26% 


.33 


.37* 


203 


Percentage of screen area for 
information 


67% 


22% 


.38* 


.48** 


215 


Keys PgUp PgDn 


.66 


.49 


.29 


.57** 


319 


Responding in the assessment with 
keyboard 


.39 


.50 


.03 


.41* 


322 


Guidelines for responding in the 
assessment 


.57 


.50 


.23 


.55** 


325 


Maximal number of the keystrokes 
for responding in the assessment 


26.86 


43.61 


.33 


61** 



* Statistically significant at the 0.05 level ** Statistically significant at the 0.01 level. 

In the bold are given the correlation coefficients of boys’ and girls’ evaluations, which 
are different from each other at the 0.05 level. 



Table 2 Correlation coefficients between the evaluations of fitness of assessment and 



the characteristics of program units. 



Characteristic 


Mean 

value 


Standard 

deviation 


Correlation 
with boys' 
evaluation 


Correlation 
with girls' 
evaluation 


No 


Name 


300 


Questions about solely unit in 
the assessment 


.36 


.49 


.51** 


.39* 


318 


Essential questions in the 
assessment 


.76 


.43 


60** 


.56** 


319 


Responding in the assessment 
with keyboard 


.39 


.50 


.17 


.48** 


322 


Guidelines for responding in the 
assessment 


.57 


.50 


.19 


.47* 


325 


Maximal number of the 
keystrokes for responding in 
the assessment 


26.86 


43.61 


.03 


.55** 


338 


Announcement of the 
percentage of right answers 


.57 


.50 


.21 


.65** 


340 


Announcement of the 
responding time 


.36 


.49 


-.39* 


.06 


342 


New trial after the wrong 
answer in the assessment 


.27 


.46 


.79 


.30 


346 


Commendation after the right 
answer in the assessment 


.27 


.46 


.79 


.30 



* Statistically significant at the 0.05 level ** Statistically significant at the 0.01 level. 

In the bold are given the correlation coefficients of boys’ and girls’ evaluations, which 
are different from each other at the 0.05 level. 












Discussion 

The girls’ evaluations of the plainness of the manipulating were correlated 
with the 17 characteristics of the manipulation of the program. The boys’ evaluations 
of the plainness of the manipulation however were correlated only with the one 
characteristic. The reason for that result would be the fact that boys in our study had 
significantly higher computer skills and less computer anxiety than girls before the 
experiment (Mann-Whitney U-test p<.05) according to the questionnaires. The higher 
computer skills and lesser anxiety of boys ensure that boys can handle different 
programs’ manipulation. 

We found that the girls preferred more guidelines for the manipulation of the 
program. Units of the multimedia textbooks which had guidelines how to continue on 
the title page, how to input answers in assessment, were rated more highly by the 
girls. 

Also, the girls preferred simpler program interface. A smaller number of key- 
combinations helps to reduce the navigational difficulty. Key-combinations are more 
difficult to remember than different buttons and icons. Also, the number of navigation 
possibilities and number of commands (icons, buttons, key-combinations etc) were 
negatively correlated with the girls’ evaluations. If students can navigate with a 
mouse, with keys, with buttons and with menus, it is deceptive for learners and they 
do not know how to manipulate the program. Also Alessi and Trollip (2001, 173) 
recommend avoiding a greater number of navigation’ possibilities. Our research 
showed that the girls preferred the familiar icons and commands. We considered the 
commands and the buttons in the units which were the same as in MS Office 
programs and Internet Explorer as the familiar ones. The latter programs are taught in 
Estonian schools. Higher computer skills and lesser anxiety of boys ensured that boys 
could handle the more difficult program manipulation and were able to navigate the 
program even when they did not know exactly what the command or icon meant. 
Boling et al. (1998) wrote that novice users should interpret more icons than 
experienced users, who recognize familiar buttons quickly and easily. Amber (2000) 
recommended taking over standards from IBM and Microsoft. Users are used to these 
buttons and icons and these corporations have defined about 95-99% icons, buttons, 
menus etc, which are needed in the user’s interface. 

The better computer skills of boys insure also that they do not get lost in the 
multimedia materials. The girls in our experiment needed more help for orientation. 




They preferred less numbers of the levels in the menus and hyperlinks with marking, 
but the search capability and terms in the words of sub-menus were not preferred by 
the girls. All these four characteristics are connected somehow with the orientation. 
Levels in the hierarchical menus hinder the information and less experienced user do 
not find the necessary material. Alessi and Trollip (2001) recommend keeping the 
levels of the hierarchical menus few in number. Also the terms in the sub-menus 
hinder the information. When the user does not know the meaning of the choice in the 
menus or in the sub-menus he/she does not choose it. Hyperlinks with marking mean 
that hyperlink change colour if once selected. This kind of feature prevents going in 
circles when the user is lost in hyperspace. Search capability is useful when the 
information seeking is needed, but when the goal is obtaining the textual material, 
inexperienced users may get lost, because they do not know how they got to the 
concrete page and how they can go back. 

An interesting result was that the girls’ evaluations of the plainness of the 
manipulation of the program unit were negatively correlated with the attractiveness of 
realization, evaluated by the experts. The reason for that result might be again the fact 
that the girls in our study had less computer experiences. Mayer and Moreno (2002) 
and Najjar (2001) point out that effects of the multimedia presentation have more 
influence on the inexperienced users and lead their attention away from the learning 
goal. Therefore the great attractiveness of the realization might lead the girls’ 
attention away from the manipulation too. 

Manipulation was simpler for the girls with the keys. The girls preferred to 
navigate in the material with the keys Page Up and Page Down and they preferred 
also to input answers in the assessment with the keyboard. Also, the maximal number 
of keystrokes for responding in the assessment positively related to the girls’ 
evaluations of the plainness of the manipulation. 

The percentage of the screen area for text and the percentage of the screen area 
for the information are not exactly the manipulation characters but they have an 
influence on navigation. When the program window covers only a half area of the 
screen, all the other programs’ icons and the office toolbar are still visible and may 
distract attention. Also the inexperienced user may unintentionally click on other 
icons and he/she gets frustrated when another program opens. Therefore the 
percentage of the screen area for the information was positively correlated with both 




the boys’ and girls’ evaluations of the manipulation. The percentage of the screen area 
for the text was positively correlated with the girls’ evaluations. 

But there was not any evidence that concrete structure of the unit (linear, 
hierarchical or unstructured) is easier to navigate for the boys than for the girls (with 
ANOVA p>.05). 

About the fitness of the assessment as boys, so girls need the questions to be 
about single unit, which they have just studied and questions had to be essential. 
Replying is simpler for the girls’ with the keys. Responding with the keyboard in the 
assessment and maximal number of keystrokes for responding were positively 
correlated with the girls’ evaluations of the fitness of assessment. And as we have 
repeatedly mentioned the girls in our study had less computer skills, so they needed 
guidelines for assessment. 

There were statistically significant differences in students’ evaluations of the 
fitness of assessment by the features of assessment (with ANOVA pc. 01). Tukey’s 
HSD indicated that the boys’ evaluations were significantly higher if the assessment 
was a part of the unit (pc.001). In became evident from the girls’ evaluations that 
their evaluations for assessment as a part of the unit were significantly higher than for 
assessment without feedback. The reason for that result might be that the girls need 
more feedback than boys as was suggested by Chanlin (1999). 

The most important differences were found in the boys’ and girls’ evaluations 
about the feedback. The boys preferred new trial when the answer was wrong, and 
feedback after the right answer with commendation. Announcement of the responding 
time was negatively correlated with the boys’ evaluations. The girls preferred 
feedback, which could inform them about the percentage of right answers. 

Conclusion 

The lists of preferred characteristics were different for girls and boys. The girls 
need simpler manipulation of the programs than boys do. To design educational 
software which motivates girls, the complicacy of navigation and guidelines for 
orientation must be carefully controlled. As the assessment is evaluated more highly if 
it is a part of a learning unit, it is more rational not to program the assessment as a 
separate module or piece of software. When designing the assessment, preferences of 
feedback both boys and girls should be taken into consideration. 
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